{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 作业三：\n",
    "写一个可以将MNIST图片向任意方向（上，下，左，右）移动一个像素功能。然后对训练集中的每张图片，创建四个位移后的副本，每个方向一个，添加到训练集。最后，在这个扩展过的训练集上训练模型，衡量其在测试集上的精度，来优化精度，这种人工扩展训练集的技术成为数据增广或训练集扩展 15分"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 思路\n",
    "- 获取数据:done\n",
    "- 获取样本X,标签y；将X，y的顺序随机打乱：done\n",
    "- 获取训练集60000；测试集10000：done\n",
    "- 获取一个样本数据：39000：？，之后测试用：done\n",
    "- 样本数据shape(28,28),(上下左右)移动一个像素\n",
    "- 训练集都移动一个像素\n",
    "- 位移副本添加到训练集：np.c_\n",
    "- 使用模型：\n",
    "    - SGD梯度下降 分类器\n",
    "    - 支持向量机svm\n",
    "    - 随机森林和朴素贝叶斯处理多类别分类器（这个例子不使用）\n",
    "- 交叉验证取得precision/recall\n",
    "- "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 要用于增加训练数据集，让数据集尽可能的多样化，使得训练的模型具有更强的泛化能力"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import fetch_mldata\n",
    "import numpy as np\n",
    "import pandas as pd \n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据集读取\n",
    "mnist = fetch_mldata('mnist-original', data_home='./')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 数据\n",
    "X=mnist['data']\n",
    "y=mnist['target']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 洗牌 \n",
    "shuffle_index=np.random.permutation(70000)\n",
    "# 数据\n",
    "X=X[shuffle_index]\n",
    "# 标签\n",
    "y=y[shuffle_index]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(70000, 784) (70000,)\n"
     ]
    }
   ],
   "source": [
    "print(X.shape,y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 训练集，测试集拆分\n",
    "X_train,y_train,X_test,y_test=X[:60000,:],y[:60000],X[60000:,:],y[60000:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# data_new=X_train.reshape((len(X_train),int(np.sqrt(X_train.shape[1])),-1))\n",
    "# data_new.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import matplotlib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAADGpJREFUeJzt3V+IXPUZxvHnqTZeGEFDxhiNdtMa\nSkXTpIxJRRFLtdgSiQpKc1FSDE0vKlTJRYMgilAQqdoKUklraIqNbbA15kLaiBQ0UCUbEY1J24hs\nTZqwmaBSmwv/5e3FnpRt3DkzmTkzZ9L3+4Flzpz3zDkvwz57ZuZ3Zn+OCAHI5zN1NwCgHoQfSIrw\nA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBSpw/zYHPnzo2xsbFhHhJIZWJiQkeOHHE32/YVftvXS/qZ\npNMk/TIi7i/bfmxsTOPj4/0cEkCJZrPZ9bY9v+y3fZqkRyV9U9IlklbZvqTX/QEYrn7e8y+T9GZE\nvBURH0r6raSV1bQFYND6Cf8FkvZPu3+gWPc/bK+1PW57vNVq9XE4AFXqJ/wzfajwqe8HR8SGiGhG\nRLPRaPRxOABV6if8ByRdOO3+AkkH+2sHwLD0E/6dkhbZXmh7lqRvS9pWTVsABq3nob6I+Nj27ZL+\npKmhvo0R8UZlnQEYqL7G+SPiWUnPVtQLgCHi8l4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kR\nfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJ\nEX4gKcIPJEX4gaQIP5AU4QeS6muWXtsTkt6X9ImkjyOiWUVTAAavr/AXvhYRRyrYD4Ah4mU/kFS/\n4Q9J223vsr22ioYADEe/L/uvjIiDts+V9Jztv0bEC9M3KP4orJWkiy66qM/DAahKX2f+iDhY3B6W\n9LSkZTNssyEimhHRbDQa/RwOQIV6Dr/tM22fdXxZ0jck7a6qMQCD1c/L/nmSnrZ9fD+bI+KPlXQF\nYOB6Dn9EvCXpyxX2AmCIGOoDkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrw\nA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRVxSy9qNnRo0fb1g4ePNjXvhcu\nXFhaP/308l+hffv29XX8MmeccUZpnenhynHmB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkOo7z294o\naYWkwxFxabFujqTfSRqTNCHp1oh4d3BtosyWLVva1m677ba+9n3DDTeU1juNtT/11FN9Hb/M2Wef\nXVq/9tpre973mjVrSuvz5s0rrTcajdL6ggULTrqnqnVz5v+VpOtPWLde0vMRsUjS88V9AKeQjuGP\niBckvXPC6pWSNhXLmyTdWHFfAAas1/f88yLikCQVt+dW1xKAYRj4B36219oetz3earUGfTgAXeo1\n/JO250tScXu43YYRsSEimhHR7PQhCIDh6TX82yStLpZXS3qmmnYADEvH8Nt+UtJfJH3R9gHbayTd\nL+k62/skXVfcB3AK6TjOHxGr2pS+XnEvaGP37t2l9XXr1vW878svv7y0/t577/W87046XSOwePHi\n0vrOnTtL6/1cY9Dv9QljY2Ol9bLrL+6+++6+jt0trvADkiL8QFKEH0iK8ANJEX4gKcIPJMW/7j4F\nPPLII6X1d99t/23q8847r/SxW7duLa2ff/75pfVOduzY0bY2a9as0sdedtllpfVdu3b11BOmcOYH\nkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQY5z8F7N+/v7ReNpY/6HH8Tq666qpTct8ZcOYHkiL8QFKE\nH0iK8ANJEX4gKcIPJEX4gaQY5x8Be/bsKa2/+OKLpfWrr766bW358uU99YT/f5z5gaQIP5AU4QeS\nIvxAUoQfSIrwA0kRfiCpjuP8tjdKWiHpcERcWqy7V9L3JLWKze6KiGcH1eT/uw8++KC0fvTo0SF1\ngky6OfP/StL1M6x/OCKWFD8EHzjFdAx/RLwg6Z0h9AJgiPp5z3+77ddsb7R9TmUdARiKXsP/c0lf\nkLRE0iFJD7bb0PZa2+O2x1utVrvNAAxZT+GPiMmI+CQijkn6haRlJdtuiIhmRDQbjUavfQKoWE/h\ntz1/2t2bJO2uph0Aw9LNUN+Tkq6RNNf2AUn3SLrG9hJJIWlC0vcH2COAAegY/ohYNcPqxwfQC4Ah\n4go/ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUkzRPQK2\nb99edwtIiDM/kBThB5Ii/EBShB9IivADSRF+ICnCDyTFOP8QTE5OltYfe+yxvva/YsWKvh6PnDjz\nA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBSHcf5bV8o6deSzpN0TNKGiPiZ7TmSfidpTNKEpFsj4t3B\ntTq6Pvroo9L6Sy+9VFqfmJgorV988cWl9SuuuKK0DsykmzP/x5LWRcSXJH1V0g9sXyJpvaTnI2KR\npOeL+wBOER3DHxGHIuKVYvl9SXslXSBppaRNxWabJN04qCYBVO+k3vPbHpO0VNLLkuZFxCFp6g+E\npHOrbg7A4HQdftuzJf1e0h0R8a+TeNxa2+O2x1utVi89AhiArsJv+7OaCv5vIuIPxepJ2/OL+nxJ\nh2d6bERsiIhmRDQbjUYVPQOoQMfw27akxyXtjYiHppW2SVpdLK+W9Ez17QEYlG6+0nulpO9Iet32\nq8W6uyTdL2mL7TWS3pZ0y2BaHH2PPvpoaf3OO+8src+ePbu0/sQTT5TWly5dWloHZtIx/BGxQ5Lb\nlL9ebTsAhoUr/ICkCD+QFOEHkiL8QFKEH0iK8ANJ8a+7u7R169a2tQceeKD0sYsXLy6tr19f/oXI\n5cuXl9aBXnDmB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkGOfv0ubNm9vW7rvvvtLH3nzzzaX1OXPm\n9NQT0A/O/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOP8XdqyZUvdLQCV4swPJEX4gaQIP5AU4QeS\nIvxAUoQfSIrwA0l1DL/tC23/2fZe22/Y/mGx/l7b/7T9avHzrcG3C6Aq3Vzk87GkdRHxiu2zJO2y\n/VxRezgifjK49gAMSsfwR8QhSYeK5fdt75V0waAbAzBYJ/We3/aYpKWSXi5W3W77NdsbbZ/T5jFr\nbY/bHm+1Wn01C6A6XYff9mxJv5d0R0T8S9LPJX1B0hJNvTJ4cKbHRcSGiGhGRLPRaFTQMoAqdBV+\n25/VVPB/ExF/kKSImIyITyLimKRfSFo2uDYBVK2bT/st6XFJeyPioWnr50/b7CZJu6tvD8CgdPNp\n/5WSviPpdduvFuvukrTK9hJJIWlC0vcH0iGAgejm0/4dkjxD6dnq2wEwLFzhByRF+IGkCD+QFOEH\nkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSMoRMbyD2S1J/5i2aq6kI0Nr4OSM\nam+j2pdEb72qsrfPRURX/y9vqOH/1MHt8Yho1tZAiVHtbVT7kuitV3X1xst+ICnCDyRVd/g31Hz8\nMqPa26j2JdFbr2rprdb3/ADqU/eZH0BNagm/7ett/832m7bX19FDO7YnbL9ezDw8XnMvG20ftr17\n2ro5tp+zva+4nXGatJp6G4mZm0tmlq71uRu1Ga+H/rLf9mmS/i7pOkkHJO2UtCoi9gy1kTZsT0hq\nRkTtY8K2r5b0b0m/johLi3UPSHonIu4v/nCeExE/GpHe7pX077pnbi4mlJk/fWZpSTdK+q5qfO5K\n+rpVNTxvdZz5l0l6MyLeiogPJf1W0soa+hh5EfGCpHdOWL1S0qZieZOmfnmGrk1vIyEiDkXEK8Xy\n+5KOzyxd63NX0lct6gj/BZL2T7t/QKM15XdI2m57l+21dTczg3nFtOnHp08/t+Z+TtRx5uZhOmFm\n6ZF57nqZ8bpqdYR/ptl/RmnI4cqI+Iqkb0r6QfHyFt3paubmYZlhZumR0OuM11WrI/wHJF047f4C\nSQdr6GNGEXGwuD0s6WmN3uzDk8cnSS1uD9fcz3+N0szNM80srRF47kZpxus6wr9T0iLbC23PkvRt\nSdtq6ONTbJ9ZfBAj22dK+oZGb/bhbZJWF8urJT1TYy//Y1Rmbm43s7Rqfu5GbcbrWi7yKYYyfirp\nNEkbI+LHQ29iBrY/r6mzvTQ1ienmOnuz/aSkazT1ra9JSfdI2ippi6SLJL0t6ZaIGPoHb216u0ZT\nL13/O3Pz8ffYQ+7tKkkvSnpd0rFi9V2aen9d23NX0tcq1fC8cYUfkBRX+AFJEX4gKcIPJEX4gaQI\nP5AU4QeSIvxAUoQfSOo/wj55bVrvAkoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a6a86bab00>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 图片展示\n",
    "%matplotlib inline\n",
    "some_digit=X_train[40000]\n",
    "some_digit_img=some_digit.reshape(28,28)\n",
    "plt.imshow(some_digit_img,cmap=matplotlib.cm.binary)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "def image_offset(data_s,dir='u'):\n",
    "\n",
    "    size=len(data_s)\n",
    "    en_ret = np.zeros((size, 784))\n",
    "    if direc == 'u':        \n",
    "        for i in range(size):\n",
    "            trans_data = np.append(data_s[i][1:,:], data_s[i][0:1,:],axis=0)\n",
    "            #print(trans_data.shape)\n",
    "            en_ret[i] = trans_data.reshape(1, -1)\n",
    "    elif direc == 'd':\n",
    "        for i in range(size):\n",
    "            trans_data = np.append(data_s[i][-1:,:], data_s[i][:-1,:],axis=0)\n",
    "            #print(trans_data.shape)\n",
    "            en_ret[i] = trans_data.reshape(1, -1)\n",
    "    elif direc == 'l':\n",
    "        for i in range(size):\n",
    "            trans_data = np.append(data_s[i][:,1:], data_s[i][:,0:1],axis=1)\n",
    "            #print(trans_data.shape)\n",
    "            en_ret[i] = trans_data.reshape(1, -1)\n",
    "    elif direc == 'r':\n",
    "        for i in range(size):\n",
    "            trans_data = np.append(data_s[i][:,-1:], data_s[i][:,:-1],axis=1)\n",
    "            #plt.imshow(trans_data, cmap = matplotlib.cm.binary,interpolation=\"nearest\")\n",
    "            en_ret[i] = trans_data.reshape(1, -1)\n",
    "    return en_ret\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'direc' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-36-3fd7fadfe31d>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mX_train_u\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mimage_offset\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mdir\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'u'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      2\u001b[0m \u001b[0mX_train_d\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mimage_offset\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mdir\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'd'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[0mX_train_l\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mimage_offset\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mdir\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'l'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[0mX_train_r\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mimage_offset\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_train\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mdir\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'r'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m<ipython-input-35-d9ef0d2474ce>\u001b[0m in \u001b[0;36mimage_offset\u001b[1;34m(data_s, dir)\u001b[0m\n\u001b[0;32m      3\u001b[0m     \u001b[0msize\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mlen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata_s\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m     \u001b[0men_ret\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mzeros\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msize\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m784\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 5\u001b[1;33m     \u001b[1;32mif\u001b[0m \u001b[0mdirec\u001b[0m \u001b[1;33m==\u001b[0m \u001b[1;34m'u'\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      6\u001b[0m         \u001b[1;32mfor\u001b[0m \u001b[0mi\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0msize\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      7\u001b[0m             \u001b[0mtrans_data\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata_s\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mi\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdata_s\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mi\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m0\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0maxis\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;36m0\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mNameError\u001b[0m: name 'direc' is not defined"
     ]
    }
   ],
   "source": [
    "X_train_u=image_offset(X_train,dir='u')\n",
    "X_train_d=image_offset(X_train,dir='d')\n",
    "X_train_l=image_offset(X_train,dir='l')\n",
    "X_train_r=image_offset(X_train,dir='r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train_new=np.concatenate((X_train,X_train_u,X_train_d,X_train_l,X_train_r),axis=0)\n",
    "y_train_new=np.concatenate((y_train,y_train,y_train,y_train,y_train),axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 实例化 \n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "kn_clf=KNeighborsClassifier()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 填充数据\n",
    "kn_clf.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 验证\n",
    "kn_clf.predict([some_digit])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import precision_score,recall_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import cross_val_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import cross_val_predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "y_train_5=(y_train.astype(int)==5)\n",
    "y_train_5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "kn_clf.fit(X_train,y_train_5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_train_pred=cross_val_predict(kn_clf,X_train,y_train_5,cv=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 精度，召回\n",
    "precision=precision_score(y_train_5,y_train_pred)\n",
    "recall=recall_score(y_train_5,y_train_pred)\n",
    "print('precision:{:.1f}%'.format(precision*100))\n",
    "print('recall:{:.1f}%'.format(recall*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "kn_clf_new=KNeighborsClassifier()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_train_new_5=(y_train_new.astype(int)==5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "kn_clf_new.fit(X_train_new,y_train_new_5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "kn_clf_new.predict([some_digit])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_train_new_pred=cross_val_predict(kn_clf_new,X_train_new,y_train_new_5,cv=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "precision=precision_score(y_train_new_5,y_train_new_pred)\n",
    "recall=recall_score(y_train_new_5,y_train_new_pred)\n",
    "print('precision:{:.1f}%'.format(precision*100))\n",
    "print('recall:{:.1f}%'.format(recall*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_test_5=(y_test.astype(int)==5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_test_pred=cross_val_predict(kn_clf_new,X_test,y_test_5,cv=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "precision=precision_score(y_test_5,y_test_pred)\n",
    "recall=recall_score(y_test_5,y_test_pred)\n",
    "print('precision:{:.1f}%'.format(precision*100))\n",
    "print('recall:{:.1f}%'.format(recall*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 结论\n",
    "- 没有进行数据增广的表现：\n",
    "    - precision:97.0%\n",
    "    - recall:95.5%\n",
    "- 像素位移后，模型表现\n",
    "    - precision:98.0%\n",
    "    - recall:97.2%\n",
    "- 测试集表现\n",
    "    - precision:95.3%\n",
    "    - recall:91.1%\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
